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ABSTRACT 


Background. The CDC recently defined being “up-to-date” on COVID-19 vaccination as having 
received at least one dose of a COVID-19 bivalent vaccine. The purpose of this study was to compare the 
risk of COVID-19 among those “up-to-date” and “not up-to-date” on COVID-19 vaccination. 

Methods. Employees of Cleveland Clinic in employment when the COVID-19 bivalent vaccine first 
became available, and still employed when the XBB lineages became dominant, were included. 
Cumulative incidence of COVID-19 since the XBB lineages became dominant was compared across the 
*up-to-date” and “not up-to-date” states, by treating COVID-19 bivalent vaccination as a time-dependent 
covariate whose value changed on receipt of the vaccine. Risk of COVID-19 by vaccination status was 
also compared using multivariable Cox proportional hazards regression adjusting for propensity to get 
tested for COVID-19, age, sex, and phase of most recent prior SARS-CoV-2 infection. 

Results. COVID-19 occurred in 1475 (3%) of 48 344 employees during the 100-day study period. 
The cumulative incidence of COVID-19 was lower in the “not up-to-date” than in the “up-to-date” state. 
On multivariable analysis, not being “up-to-date” with COVID-19 vaccination was associated with lower 
risk of COVID-19 (HR, 0.77; 95% C.I., 0.69-0.86; P-value, <0.001). Results were very similar when those 
65 years and older were only considered “up-to-date” after receiving 2 doses of the bivalent vaccine. 
Conclusions. Since the XBB lineages became dominant, adults “not up-to-date” by the CDC definition 
have a lower risk of COVID-19 than those “up-to-date” on COVID-19 vaccination, bringing into question 


the value of this risk classification definition. 
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INTRODUCTION 


In April 2023 the Centers for Disease Control and Prevention (CDC) updated its guidance on 
Coronavirus Disease 2019 (COVID-19) vaccination to consider all individuals above the age of 6 to be 
“up-to-date” with COVID-19 vaccination only if they had received at least one dose of a COVID-19 
bivalent vaccine [1]. By this definition, those who had not received a single dose of a COVID-19 bivalent 
vaccine would be considered not “up-to-date”. A recent study was unable to find the bivalent vaccine to be 
effective while the XBB variants were the dominant circulating strains [2]. Given this lack of effectiveness 
of the bivalent vaccine against the XBB variants, which are the predominant circulating strains at this time, 
it is reasonable to question whether individuals “up-to-date” with a vaccine of questionable effectiveness 
are protected against COVID-19 compared to those not “up-to-date”. 

The purpose of this study was to evaluate whether individuals who were not “up-to-date” on 


COVID-19 vaccination had a higher risk of COVID-19 than those “up-to-date”. 
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METHODS 


Study Design 


This was a retrospective cohort study conducted at the Cleveland Clinic Health System (CCHS) in 


the United States. 


Patient Consent Statement 


The study was approved by the Cleveland Clinic Institutional Review Board as exempt research 
(IRB no. 22-917). A waiver of informed consent and waiver of HIPAA authorization were approved to 


allow the research team access to the required data. 


Setting 


Since the arrival of the COVID-19 pandemic at Cleveland Clinic in March 2020, employee access 
to testing has been a priority. Voluntary vaccination for COVID-19 began on 16 December 2020, and the 
monovalent mRNA vaccine as a booster became available to employees on 5 October 2021. The COVID- 
19 bivalent mRNA vaccine began to be offered to employees on 12 September 2022. The study start date 


was 29 January 2023, the date the XBB lineages first became the dominant circulating strains in Ohio. 


Participants 


CCHS employees in employment at any location in Ohio when the COVID-19 bivalent vaccine 
first became available (12 September 2022), and who were still employed when the XBB lineages became 
dominant (29 January 2023), were included in the study. Those for whom age and sex were not available 


were excluded. 
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Variables 


Covariates collected were age, sex, job location, and prior SARS-CoV-2 infection, as described in 
our earlier studies [2-5]. Institutional data governance rules related to employee data limited our ability to 
supplement our dataset with additional clinical variables. Vaccination status was “up-to-date” or “not up- 
to-date” based on whether or not a person had received at least one dose of a COVID-19 bivalent vaccine. 
For each subject, the propensity to get tested for COVID-19 was defined as the number of COVID-19 
nucleic acid amplification tests (NAATs) done divided by the number of years of employment at CCHS 
during the pandemic. The pandemic phase during which a subject had his or her last prior episode of 
COVID-19 was also collected as a variable, based on which variant/lineages accounted for more than 50% 


of infections in Ohio at the time [6]. 


Outcome 


The study outcome was time to COVID-19, the latter defined as a positive NAAT for Severe 
Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) any time after the study start date. Outcomes 
were followed until 10 May 2023, allowing for evaluation of outcomes up to 100 days from the study start 


date. 


Statistical Analysis 


A Simon-Makuch hazard plot [7] was created to compare the cumulative incidence of COVID-19 
in the “up-to-date” and “not up-to-date” states, by treating vaccination status as a time-dependent 
covariate. A subject’s vaccination status was “not up-to-date” before receipt of a COVID-19 bivalent 
vaccine, and “up-to-date” after receipt of the vaccine. Subjects whose employment was terminated during 


the study period before they had COVID-19 were censored on the date of termination. Curves for the “not 


medRxiv preprint doi: https://doi.org/10.1101/2023.06.09.23290893; this version posted June 12, 2023. The copyright holder for this preprint 
(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. 
It is made available under a CC-BY-NC-ND 4.0 International license . 


up-to-date” state were based on data while the vaccination status was “not up-to-date”. Curves for the “up- 
to-date” state were based on data from the date the vaccination status changed to “up-to-date”. 

Multivariable Cox proportional hazards regression models were fit to examine the association of 
various variables with time to COVID-19. Vaccination status was included as a time-dependent covariate 
[8]. The possibility of multicollinearity in the models was evaluated using variance inflation factors. The 
proportional hazards assumption was checked using log(-log(survival)) vs. time plots. The hazard ratio for 
vaccination status provided a comparison of risk of COVID-19 based on vaccination status after adjusting 
for all other covariates. 

The analysis was performed by N. K. S. and A.S.N. using the survival package and R version 4.2.2 


(R Foundation for Statistical Computing) [8—10]. 
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RESULTS 


Of 48 344 included subjects, 1445 (3%) were censored during the study because of termination of 
employment. By the end of the study, 12 841(27%) were “up-to-date” on COVID-19 vaccination, of whom 
11 187 (87%) received the Pfizer vaccine and 1654 (13%) received the Moderna vaccine. A total of 1475 


employees (3%) acquired COVID-19 during the 100 days of the study. 


Baseline Characteristics 


Table 1 shows the characteristics of subjects included in the study. Notably, this was a relatively 
young population, with a mean age of 43 years. Among these, 22 407 (46%) had previously had a 
documented episode of COVID-19 and 16 262 (34%) had previously had an Omicron variant infection. 42 
160 subjects (87%) had previously received at least one dose of vaccine and 44 432 (92%) had been 
previously exposed to SARS-CoV-2 by infection or vaccination. Altogether, 36 490 subjects (76%) were 
tested for COVID-19 by a NAAT at least once while employed at Cleveland Clinic. The propensity for 
COVID-19 testing ranged from 0 to 63.5 per year, with a median of 0.64 and interquartile range spanning 


0.32 to 1.27 per year. 


Risk of COVID-19 Based on Vaccination Status and Prior Infection 


The risk of COVID-19 was lower in the “not up-to-date” state than in the “up-to-date” state, with 
respect to COVID-19 vaccination (Figure 1). When stratified by tertiles of propensity to get tested for 
COVID-19, the “not up-to-date” state was not associated with a higher risk of COVID-19 than the “up-to- 
date” state in any tertile (Figure 2). 

In contrast, consideration of prior infection provided a more accurate classification for risk of 
COVID-19. Those whose last prior SARS-CoV-2 infection occurred during the Omicron BQ or 


BA.4/BA.5 dominant phases had a substantially and significantly lower risk of COVID-19 than those 
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whose last SARS-CoV-2 infection occurred in the early Omicron or pre-Omicron periods or those not 
previously known to have had COVID-19 (Figure 3). When stratified by most recent prior SARS-CoV-2 
infection, there was no difference in risk of COVID-19 between the “up-to-date” and “not up-to-date” 
states within each most recent prior infection group, except for those not previously known to be infected, 
among whom the “up-to-date” state was associated with a higher risk of COVID-19 than the “not up-to- 


date” state (Figure 4). 


Multivariable Analysis 


In a multivariable Cox proportional hazards regression model, adjusted for propensity to get tested 
for COVID-19, age, sex, and phase of most recent SARS-CoV-2 infection, not being “up-to-date” on 
COVID-19 vaccination was associated with a lower risk of COVID-19 (HR, 0.77; 95% C.I., 0.69-0.86; P- 
value, <0.001). Having had their last prior episode of COVID-19 while lineages of the Omicron variant 
were dominant was protective against COVID-19 with higher protection the more recent the infection. 
Point estimates of hazard ratios and associated 95% confidence intervals for the variables included in the 
unadjusted and adjusted Cox proportional hazards regression models are shown in Table 2. Results were 
very similar if those 65 years and older were only considered “up-to-date” after receipt of at least 2 doses 
of a bivalent vaccine (Supplementary Table 1). Results were also very similar if an individual was only 


considered “up-to-date” 7 days after receipt of a bivalent vaccine (Supplementary Table 2). 
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DISCUSSION 


This study found that not being “up-to-date” on COVID-19 vaccination, using the current CDC 
definition, was associated with a lower risk of COVID-19 than being “up-to-date”, while the XBB lineages 
were the dominant circulating strains of SARS-CoV-2. 

There are two reasons why not being “up-to-date” on COVID-19 vaccination by the CDC 
definition was associated with a lower risk of COVID-19. The first is that the bivalent vaccine was 
somewhat effective against strains that were more similar to the strains on the basis of which the bivalent 
vaccine was developed, but is not effective against the XBB lineages of the Omicron variant [2]. The 
second is that the CDC definition does not consider the protective effect of immunity acquired from prior 
infection. Because the COVID-19 bivalent vaccine provided some protection against the BA.4/BA.5 and 
BQ lineages [2], those “not-up-to-date” were more likely than those “up-to-date” to have acquired a 
BA.4/BA.5 or BQ lineage infection when those lineages were the dominant circulating strains. It is now 
well-known that SARS-CoV-2 infection provides more robust protection than vaccination [4,11,12]. 
Therefore it is not surprising that not being “up-to-date” according to the CDC definition was associated 
with a higher risk of prior BA.4/BA.5 or BQ lineage infection, and therefore a lower risk of COVID-19, 
than being “up-to-date”, while the XBB lineages were dominant. 

The strengths of our study include its large sample size, and its conduct in a healthcare system that 
devoted resources to have an accurate accounting of who had COVID-19, when COVID-19 was 
diagnosed, who received a COVID-19 vaccine, and when. The study methodology, treating vaccination 
status as a time-dependent covariate, allowed for determining vaccine effectiveness in real time. Adjusting 
for the propensity to get tested for COVID-19 should have mitigated against concern that individuals who 
bothered to remain up-to-date on COVID-19 vaccination may have been more likely to get tested for 
COVID-19 when they had symptoms. 

Our study was limited to examination of all detected infections. We were unable to distinguish 


between symptomatic and asymptomatic infections, and the rarity of severe illnesses precluded studying 
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whether being “up-to-date” decreased severity of illness. Information on prior COVID-19 would have been 
incomplete, as many asymptomatic and mildly symptomatic infections would have been missed. There 
may have been unconsidered variables that might have influenced the results. Lastly, our study was done 
in a healthcare population, and included no children and few elderly subjects, and the majority of study 
subjects would not have been immunocompromised. 

This study’s findings question the wisdom of promoting the idea that every person needs to be 
“up-to-date” on COVID-19 vaccination, as currently defined, at this time. It is often stated that the primary 
purpose of vaccination is to prevent severe COVID-19 and death. We certainly agree with this, but it 
should be pointed out that there is not a single study that has shown that the COVID-19 bivalent vaccine 
protects against severe disease or death caused by the XBB lineages of the Omicron variant. At least one 
prior study has failed to find a protective effect of the bivalent vaccine against the XBB lineages of SARS- 
CoV-2 [2]. People may still choose to get the vaccine, but an assumption that the vaccine protects against 
severe disease and death is not reason enough to unconditionally push a vaccine of questionable 
effectiveness to all adults. 

In conclusion, this study found that not being “up-to-date” on COVID-19 vaccination by the CDC 
definition was associated with a lower risk of COVID-19 than being “up-to-date”. This study highlights 
the challenges of counting on protection from a vaccine when the effectiveness of the vaccine decreases 
over time as new variants emerge that are antigenically very different from those used to develop the 
vaccine. It also demonstrates the folly of risk classification based solely on receipt of a vaccine of 


questionable effectiveness while ignoring protection provided by prior infection. 
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TABLES 


Table 1 


Table 1. Baseline Characteristics of 48 344 Employees of Cleveland Clinic in Ohio 


Characteristics Overall 
Age in years, mean (SD) 42.5 (13.2) 
Sex 
Female 35 953(74.4) 
Male 12 381 (25.6) 
Location 
Cleveland Clinic Main 19 445 (40.2) 
Cleveland area regional hospitals 11 355 (23.5) 
Ambulatory centers 8443 (17.5) 
Cleveland Clinic Akron 3996 (8.3) 
Administrative centers 3971 (8.2) 
Cleveland Clinic Medina 1124 (2.3) 
Prior documented COVID-19" 22 407 (46.4) 
Prior Omicron variant infection” 16 262 (33.6) 


Most recent prior SARS-CoV-2 infection 


No documented infection 25 927 (53.6) 
During the pre-Omicron phase 6145 (12.7) 
During the Omicron BA.1/BA.2 dominant phase 8876 (18.4) 
During the Omicron BA.4/BA.5 dominant phase 5943 (12.3) 
During the Omicron BQ dominant phase 1443 (3.0) 
Days since most recent infection®, mean (SD) 365 (238) 
Days since most recent vaccine’, mean (SD) 370 (214) 


Days since proximate SARS-CoV-2 exposure‘, mean (SD) 306 (200) 
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Data are presented as no. (%) unless otherwise indicated. 

Abbreviation: SD, standard deviation; COVID-19, Coronavirus Disease 2019; SARS-CoV-2, Severe Acute Respiratory Syndrome Coronavirus-2 
“Defined by a positive SARS-CoV-2 nucleic acid amplification test 

"Defined by a positive SARS-CoV-2 nucleic acid amplification test during a time when Omicron variants were the predominant strains in Ohio 
“Among those with prior documented COVID-19 

“Among those who received at least one dose of a COVID-19 vaccine 


“Among those with prior documented COVID-19 or who received at least one dose of a COVID-19 vaccine 
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Table 2 


Table 2. Unadjusted and Adjusted Associations With Time to COVID-19 


Variables Unadjusted HR (95% CI) P Adjusted HR (95% CI)’ P 
Vaccination status “not up-to-date” 0.78 (0.70-0.87) <0.001 0.77 (0.69-0.86) <0.001 
Propensity to get tested for COVID-19° 1.07 (1.06 — 1.09) <0.001 1.09 (1.08-1.10) <0.001 
Age 1.001 (0.997-1.005) 0.56 0.999 (0.995-1.003) 0.65 
Male sex 0.83 (0.74-0.94) 0.004 0.81 (0.71-0.91) <0.001 


Most recent prior SARS-CoV-2 infection? 


During Pre-Omicron phase 1.11 (0.96-1.29) 0.14 1.06 (0.92-1.23) 0.44 
During Omicron BA.1/BA.2 dominant 0.91 (0.80-1.04) 0.18 0.85 (0.74-0.97) 0.02 
phase 

During Omicron BA.4/BA.5 dominant 0.31 (0.24-0.40) <0.001 0.28 (0.22-0.36) <0.001 
phase 

During Omicron BQ dominant phase 0.12 (0.05-0.27) <0.001 0.09 (0.04-0.21) <0.001 


Abbreviation: COVID-19, Coronavirus Disease 2019; HR, hazard ratio; CI, confidence interval; SARS-CoV-2, Severe Acute Respiratory 
Syndrome Coronavirus-2 


“From a multivariable Cox-proportional hazards regression model. 
Time-dependent covariate 


“Calculated as number of COVID-19 nucleic acid amplification tests done per year of employment at Cleveland Clinic during the course of the 
pandemic. 


“Reference: no documented prior infection. 
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Figure 1. Simon-Makuch hazard plot comparing the cumulative incidence of COVID-19 in the “up-to- 
date” and “not up-to-date” states with respect to COVID-19 vaccination. Day zero was 29 January 2023, 
the day the XBB lineages of the Omicron variant became the dominant strains in Ohio. Point estimates and 


95% confidence intervals are jittered along the x-axis to improve visibility. 
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Figure 2. Simon-Makuch hazard plot comparing the cumulative incidence of COVID-19 in the “up-to- 
date” and “not up-to-date” states with respect to COVID-19 vaccination, stratified by tertiles of propensity 
to get tested for COVID-19. Day zero was 29 January 2023, the day the XBB lineages of the Omicron 
variant became the dominant strains in Ohio. Point estimates and 95% confidence intervals are jittered 
along the x-axis to improve visibility. Solid lines represent the “up-to-date” states while dashed lines 


represent the “not up-to-date” states. 
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Figure 3. Hazard plot comparing the cumulative incidence of COVID-19 stratified by the phase during 
which the last prior episode of COVID-19 occurred. Day zero was 29 January 2023, the day the XBB 
lineages of the Omicron variant became the dominant strains in Ohio. Point estimates and 95% confidence 
intervals are jittered along the x-axis to improve visibility. Solid lines represent those last infected before 
the Omicron variants became dominant or those not previously known to have had COVID-19, while 


dashed lines represent those last infected while the Omicron variants were dominant. 
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Figure 4. Simon-Makuch hazard plot comparing the cumulative incidence of COVID-19 in the “up-to- 
date” and “not up-to-date” states with respect to COVID-19 vaccination, stratified by the phase during 
which the last prior episode of COVID-19 occurred. Day zero was 29 January 2023, the day the XBB 
lineages of the Omicron variant became the dominant strains in Ohio. Point estimates and 95% confidence 
intervals are jittered along the x-axis to improve visibility. “Remote infection” includes infections during 
the pre-Omicron and Omicron BA.1/BA.2 dominant periods, i.e more than 218 days before the study start 


date. Solid lines represent the “up-to-date” states while dashed lines represent the “not up-to-date” states. 
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SUPPLEMENTARY TABLES 


Supplementary Table 1 


Supplementary Table 1. Adjusted Associations With Time to COVID-19 When Those Above Age 65 


Needed Two Vaccine Doses to Be Considered “Up-To-Date” 


Variables Adjusted HR (95% CI)* P 
Vaccination status “not up-to-date” 0.75 (0.67-0.84) <0.001 
Propensity to get tested for COVID-19° 1.09 (1.08-1.10) <0.001 
Age 1.000 (0.996-1.004) 0.99 
Male sex 0.81 (0.71-0.91) <0.001 


Most recent prior SARS-CoV-2 infection? 


During Pre-Omicron phase 1.06 (0.92-1.22) 0.45 
During Omicron BA.1/BA.2 dominant 0.85 (0.74-0.97) 0.01 
phase 

During Omicron BA.4/BA.5 dominant 0.28 (0.22-0.36) <0.001 
phase 

During Omicron BQ dominant phase 0.10 (0.04-0.21) <0.001 


Abbreviation: COVID-19, Coronavirus Disease 2019; HR, hazard ratio; CI, confidence interval; SARS-CoV-2, Severe Acute Respiratory 
Syndrome Coronavirus-2 


“From a multivariable Cox-proportional hazards regression model. 
>Time-dependent covariate 


“Calculated as number of COVID-19 nucleic acid amplification tests done per year of employment at Cleveland Clinic during the course of the 
pandemic. 


“Reference: no documented prior infection. 
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Supplementary Table 2 


Supplementary Table 1. Adjusted Associations With Time to COVID-19, When a Person Was 


Considered “Up-To-Date” Only After 7 Days Had Passed Since Receipt of the Bivalent Vaccine 


Variables Adjusted HR (95% CI)’ P 
Vaccination status “not up-to-date” 0.78 (0.69-0.87) <0.001 
Propensity to get tested for COVID-19° 1.09 (1.08-1.10) <0.001 
Age 0.999 (0.995-1.003) 0.65 
Male sex 0.81 (0.71-0.91) <0.001 


Most recent prior SARS-CoV-2 infection? 


During Pre-Omicron phase 1.06 (0.92-1.22) 0.44 
During Omicron BA.1/BA.2 dominant 0.85 (0.74-0.97) 0.02 
phase 

During Omicron BA.4/BA.5 dominant 0.28 (0.22-0.36) <0.001 
phase 

During Omicron BQ dominant phase 0.09 (0.04-0.21) <0.001 


Abbreviation: COVID-19, Coronavirus Disease 2019; HR, hazard ratio; CI, confidence interval; SARS-CoV-2, Severe Acute Respiratory 
Syndrome Coronavirus-2 


“From a multivariable Cox-proportional hazards regression model. 
Time-dependent covariate 


“Calculated as number of COVID-19 nucleic acid amplification tests done per year of employment at Cleveland Clinic during the course of the 
pandemic. 


“Reference: no documented prior infection. 
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